10
Rules
For
Radicals
L
il i -
Carl Malamud
have not failed, I've just found 10,000 ways
lat won't work.
Thomas Edison
10 Rules for Radicals
An address to theWWW2010 Conference
Raleigh, North Carolina, April 30, 2010
http://public.resource.org/rules/
CC-Zero, No Rights Reserved.
http:/ / creativecommons.org/publicdomain/zero/ 1 .0/
10 Rules for Radicals
Carl Malamud
1 Rules for Radicals
Thank you for your kind invitation to speak before you
today on this occasion of the 19th International World Wide
Web Conference. I would particularly like to thank Paul
Jones, a man whose sites have hosted poets and presidents
and who has given a home on the Internet to the people of
Linux and the people of Tibet. Tim O'Reilly says you should
put more into the ecosystem than you take out, and there is
no better example of this than Paul Jones and his decades
of public service to the Internet.
Before I turn to the subject of my talk, I feel I can give
you a little context about how to judge these words by
telling you about the first time I saw the World Wide Web in
action. I was visiting Geneva in 1991 because I was
interested in CERN's role as a hub for the growing net,
using X.25 to gateway to the Soviet Union and Eastern
Europe.
While I was visiting CERN, the head of networking,
Brian Carpenter, said I should go see one of the
researchers who was doing some interesting work. I went
into a dark room and a young man sat behind his spiffy
brand-new NeXT workstation and he showed me his
research project, a derivative of SGML with a little bit of
local area networking thrown in.
I politely watched Tim Berners-Lee give me a demo of
his so-called World Wide Web, but I was skeptical. It
looked nice, of course, but then anything looked nice on
the NeXT workstation, a high-priced hunk of hardware
created by a bunch of Apple refugees. Tim showed me how
with a click you could pull up a file on another computer,
but I wasn't sure this was something that would ever catch
7 Tales of Bureaucracy
on, certainly not on a global level. I distinctly remember
thinking to myself "this won't scale."
Having presented to you my credentials as a pundit, I
would like to talk to you today about some bureaucracies
I've had occasion to encounter, and some lessons I have
learned about how citizens — citizens with no official
portfolio or status — "mere" citizens if you will — how
citizens can change the way government works.
I hope these tales are more than mere war stories, I
hope to leave you today with some rules for radicals, 10
rules to apply to governing institutions as we attempt to
change their behavior.
— Bureaucracy N° 1 —
We begin in ancient times, a time so long ago that the term
broadband referred to ISDN lines which would operate at a
massive 64,000 bits per second, the speed of a leased line
— but magically switched on and off on-demand using the
ISDN "intelligent network."
This time — the late 1 980s and the early 1 990s — was a
time when the idea of a hyperlink was still considered the
mad delusion of a wild-eyed prophet named Theodor
Nelson, hence my skepticism about TimBL's research
project.
In those days, there were two kinds of networks. There
was the so-called Internet, and there were respectable
networks. The respectable networks were being defined
by international institutions such as the International
Organization for Standardization and the International
Telecommunication Union.
1 Rules for Radicals
These institutions were based in Geneva and their work
product was meant for grownups, grownups of sufficient
means that the cost of a few thousand dollars to buy
standards documents was considered not only eminently
reasonable, but absolutely necessary to the functioning of
our standards-making organizations. These grownups
worked for telephone companies like AT&T and their PTT
peers around the world, and for a few industrial concerns
like IBM and Siemens.
Asking the International Organization for
Standardization — asking ISO to give away technical
standards would be as silly as asking the restaurant to give
away the Entrecote and the Beaujolais. In this world of
many fine lunches and dinners, there was no free lunch.
In those days, I couldn't afford the Entrecote and
certainly not the Beaujolais. I was a hack, a hack in the
traditional sense of the word making my living as a writer,
a hack with a strong interest in the Internet, a network
based on open standards, a network with no kings, a
network built on the then-radical notion that standards
should be based on decision by rough consensus and rule
by working code.
While this loose-knit band of anarchists that were
defining an Internet based on open standards were free to
ignore most of the nonsense coming out of the "standards
professionals" and their "Open Systems Interconnection"
effort — there was one thing we couldn't ignore and that
was the telephone network on which we built our
unreliable, best-effort datagram service.
7 Tales of Bureaucracy
This telephone network was defined by the
International Telecommunication Union, the ITU, in the
1988 edition of a document called the Blue Book, a 19,000
page compendium that contained the standards for things
like how modems worked, how to compress audio, and the
operation of Signaling System Number 7. Anybody
defining Internet standards that interfaced effectively with
the underlying telephone network needed the Blue Book.
But, I couldn't afford the Blue Book— it cost 2,500 Swiss
Francs — and since I was making my living as a columnist
for trade press publications such as Communications
Week, since I couldn't write about what the Blue Book
contained, I instead wrote a lot of columns about how I
couldn't afford the Blue Book and why it should be free.
At the time, the Secretary-General of the ITU was a big
Finn named Pekka Tarjanne, a job he got as a reward after
a career in Finnish politics followed by 12 years as head of
the Finnish PTT. Tarjanne had hired a lawyer named
Anthony Rutkowski as his counsellor, and that was
probably a mistake as Tony Rutkowski was in reality a
double-agent, an Internet sympathizer who even used
email.
In 1991, the ITU was not exactly a technically
progressive organization. They had lots of rotary-dial
telephones of course, because their founding treaty
specified they got free phone calls from all the PTTs, a fact
they were inordinately proud of and never ceased to point
out.
But there was no email and only a single fax machine
for the entire 17-story building, and this telefacsimile
1 Rules for Radicals
device was carefully secured in a deputy secretary-
general's office and required a special form with many
signatures before a document was considered fax-worthy.
Tony Rutkowski read my columns flaming about the ITU
and he got me a meeting with Dr. Tarjanne. I flew to
Geneva and soon found myself in the rather spectacular
Secretary-General's suite on the top floor of the ITU tower.
After a few pleasantries about Finland — Reindeer,
saunas — we got down to business. I stated my case: the
Blue Book ought to be available for free on the Internet.
Dr. Tarjanne smiled the smile of a patient father and
told me that in his ideal world, the Blue Book and indeed
the Entrecote and maybe even the Beaujolais would all be
free, but this was of course impossible, as much as we both
might share this dream of an ideal world.
You see, it really wasn't about the money, Tarjanne
explained, there was a technical obstacle. The Blue Book
was being produced on an ancient mainframe using an
ancient program, a program so old that they had lost the
source code and nobody was quite sure exactly how it
worked.
They were developing a new typesetting system, but
that was several years away, and for now they were stuck
with only one output device, and that was their printing
press.
Dr. Tarjanne was sure I could see, while he'd love to
give me the source to the Blue Book, it wouldn't do me any
good, even his own expert technical staff didn't know quite
how this black box worked.
7 Tales of Bureaucracy
I suggested that perhaps we could try an experiment —
and this is my first rule for radicals: call everything you do
an experiment — and the experiment I proposed was that
perhaps the ITU could furnish a set of tapes for the Blue
Book and the Internet would try its hand at reverse-
engineering the system. If we succeeded, we'd give the
ITU back their Blue Book in some coherent ultra-modern
format like TROFF.
Since Dr. Tarjanne knew the Internet had only a few
users, none of them serious people of means who buy
standards, he'd be able to shut up the critics by saying he
had cooperated, but the Internet had not been up to the
task.
He said we could have a set of tapes.
I went to Boulder, Colorado and enlisted the help of
Mike Schwartz, a professor of computer science and the
creator of the original search engine, netfind. After a few
false starts, we managed to mount the tapes and read the
raw data into a series of octal dumps hundreds of pages
long, which we spread on the floor next to a printed copy
of the Blue Book.
By comparing the octal dump to the final form, we were
able to confirm that the Secretary-General was correct —
this system was a total mess — but after a lot of head-
scratching and a few surprises, we managed to turn their
system into TROFF and then into NROFFed ASCII and
PostScript, posted the tarballs on an FTP server, and sent a
note to the IETF list.
The next day, the National Science Foundation called,
complaining that the Blue Book release was using up all
1 Rules for Radicals
the bandwidth on the international backbones. The cross-
Atlantic line was still just an El, running at 2 megabits per
second and costing the NSF $60,000 per month, and we
were using all the bits.
I reassured the government program managers that this
was a temporary phenomenon, and soon we had saturated
our market with files spreading out to 500 hosts in 27
countries.
Then, word started to trickle back to Geneva that
maybe the Internet was a bit bigger than previously
thought, and soon after that a telefacsimile arrived from an
ITU official, who explained that he had been instructed by
the Secretary-General to convey to me a message, and that
message was that our experiment was now over. The
Secretary-General was insisting that we remove the Blue
Book from our server. Oh, and while we were at it, remove
all copies of the Blue Book from the rest of the Internet,
within 20 days.
I conveyed my kind regards to the Secretary-General,
and explained that while of course I would comply with
respect to my own server, that vis-a-vis the rest of the
world, there was not much that could be done as the
proverbial cat had escaped from the proverbial bag.
This is thus my second rule for radicals, and that is
when the authorities finally fire that starting gun — and do
something like send you tapes — run as fast as you can, so
when they get that queasy feeling in their stomach and
have second thoughts, it is too late to stop.
Tarjanne the Finn was my first real bureaucrat, but the
Blue Book underscored for me the importance of open
7 Tales of Bureaucracy
standards, that if code is law then it must surely follow that
law is code — and if that is the case, then the only way that
makes sense to release this code has to be open source.
— Bureaucracy N° 2 —
For the rest of my story today, I turn from Geneva back to
the United States.
In 1993, 1 had graduated from print to the wonderful
world of multimedia, which meant mostly 8-bit GIF files.
The blinky tag had not even been invented yet.
Most of us running network ops were using FTP, email,
and perhaps gopher and Archie. With those tools, I was
running an Internet radio station called Internet Talk
Radio. The flagship program was Geek of the Week, which
most people retrieved by launching an overnight FTP job
and then — assuming the sound card was properly installed
— listened to the sound file on their workstations.
Not everybody had FTP, and one listener used the MCI
Mail FTP gateway, which broke the 30-megabyte sound
files up into several hundred mail messages. When all the
messages arrived, he reassembled them and curled up to
his workstation for his episode of Geek of the Week.
We did a lot of "the future is here" Internet demos in
those days, and after giving one in Congress, I was called
aside by the staff of Congressman Edward Markey, and
they showed me a letter from a Nader's Raider named
Jamie Love, saying that the Securities and Exchange
Commission database of public filings of corporations —
known as EDGAR for the Electronic Data Gathering and
1 Rules for Radicals
Retrieval System — that this EDGAR database should be
available on the Internet for free.
These EDGAR filings were used by stock brokers,
economists, and analysts, and a $300 million/year industry
had sprung up retailing these documents. When I was a
doctoral student in economics, I learned that sometimes
you could write to the corporation and ask them to send
you their annual report by U.S. Mail, but often I ended up
forking over $30 a document to some information retailer
to read these filings electronically.
To feed this $300 million/year industry, the SEC had set
up a $30 million deal with Mead Data Corporation. The
theory was that these filings were indigestible raw data, so
Mead would act as the information wholesaler and add
"value" to the documents — and they would sell to
information retailers, who would add even more "value" to
the documents — and finally these documents would reach
the information consumer, presumably professionals on
Wall Street who knew how to read these highly technical
filings full of, you know, numbers and stuff.
A very brave bureaucrat at the National Science
Foundation, Dr. Steve Wolff, arranged for my nonprofit
radio station to get a few hundred thousand dollars,
enough so we could buy a feed of the EDGAR data. Eric
Schmidt, then the CTO at Sun Microsystems, pitched in a
box with four SPARC-2 processors. UUNET offered free
transit, Cisco threw in a router, and MFS Datanet provided
a 10 megabit fiber link to the Internet Exchange known as
MAE East.
7 Tales of Bureaucracy
That 10-megabit link was fast enough that when the
new Clinton administration took office, they asked to
borrow it. Turns out the new tenants over at the White
House were having trouble getting their routers cleared by
the Secret Service and they wanted to do a "we know what
the Internet" is event with the President, so ARPA helped
us run an infra-red link from the roof of the National Press
Building down to the White House lawn to get them hooked
up.
About ninety days after the NSF grant came through,
the server was up and running and raw EDGAR data was
on the net. Remember rule 2, when the starter's pistol gets
fired, run as fast as you can.
We ran the service for a year and a half, starting with
FTP for tarballs, then Gopher for docs, then an HTTP
server, then finally a WAIS database. By mid- 1995, there
were 50,000 people a day using the service.
Some of those people were financial fat cats on Wall
Street, but there were also students, journalists,
government employees, senior citizen investment clubs,
and others that were of insufficient means to afford $30
documents.
And this brings us to rule number 3, which is that
eyeballs rule. Build up a user base, and you have much
more leverage than if you're just blowing smoke.
Perhaps we should have used our "first-mover
advantage" to — as they said in the .com days — "monetize
the eyeballs." But, I didn't want to be the face of the SEC, I
wanted the SEC to do their job, which was to make the
1 Rules for Radicals
EDGAR database available to the public, on the Internet,
for free, and in an at least moderately clueful manner.
So I pulled the plug.
A sign appeared on the web server saying "this service
will terminate in 60 days. Click here for more information."
When you clicked, you got a page with source code,
usage stats, cost figures, and configurations to run the
system, and a series of "click here" links if you felt that
termination of the system would somehow inconvenience
you.
The first "click here" link was "click here to send mail
to Newt Gingrich," the hip young Speaker of the House.
The next was "click here to send mail to Al Gore," the hip
young Vice President. They both had email accounts and
were very proud of them.
The third "click here" link was "click here to send mail
to the Chairman of the SEC." Chairman Arthur Levitt, a
grand old man of finance, didn't have an email address, so
we created one for him. A couple of days later, the 17,000
messages he received were printed and delivered to the
SEC front desk.
Coincidentally, the SEC had scheduled an EDGAR
Industry Day meeting, which we weren't invited to, so we
crashed it. After some theatrics, one of the Commissioners
came up and asked some simple questions, like how much
it would cost to run the service and who the users were.
The Commissioner evidently briefed the Chairman,
because that evening Chairman Levitt called the
Associated Press and the Wall Street Journal and said the
SEC was going to offer this EDGAR database on the
7 Tales of Bureaucracy
Internet. The filings weren't a product, they were the glue
that make our financial markets work efficiently by
requiring corporations to disclose information to the
public.
The next day, the chief of staff called up and said that
while he fully supported his Chairman, there was one hitch
and that was that there was no way they could buy a
computer in 60 days and besides, their Internet line had
been installed but didn't seem to be working. Could we
extend the deadline? I said the deadline was firm.
The chief of staff ended up signing a loaner agreement,
we put some Sun boxes in the back of a station wagon,
drove down to SEC headquarters and helped them
configure their Cisco router and Tl line.
They were up and running by the deadline. The
computer staff ended up tickled pink they were running
the U.S. government's busiest web server and getting tons
of fan mail from their adoring public.
Rule 4 is that when you achieve your objective, don't be
afraid to turn on a dime and be nice. You can bang the
table and be a total pain in the ass, but there comes a time
to be helpful, courteous, and friendly.
— Bureaucracy N° 3 —
For the next bureaucracy, let's fast-forward this Wayback
Machine to late 2006 and early 2007 — when Google
bought YouTube for $1.65 billion, when C-SPAN started
allowing you to use their video of congress on your blog,
the year Netflix started streaming videos — the year "you"
1 Rules for Radicals
were named Time Man of the Year. This is when video
came to the Internet.
Well, not all the Internet.
Back in Washington, D.C. was an agency called the
National Technical Information Service, NTIS, a
government profit center tasked with, among other things,
being the official retailer of videos from all across the
government.
A look at the NTIS web site showed thousands of videos
from 54 different federal agencies. There was all sorts of
useful stuff — none of it viewable on the Internet — like
training materials for volunteer firefighters from the U.S.
Fire Academy.
But the prices! Ooh la la! Talk about champagne wishes
and caviar dreams! An Ellis Island documentary — "Island
of Hope, Island of Tears"— cost $55 for a 29-minute VHS
tape.
"The Time of Apollo" from NASA? $50 for 28 minutes.
I forked over $336, ordered some tapes, and posted
them to YouTube and the Internet Archive. "John F.
Kennedy: Years of Lightening," from the U.S. Information
Agency. "Firefighter Safety and Survival" from the U.S. Fire
Academy, and "Day of the Killer Tornados" from FEMA.
The nice thing about the U.S. government is pretty
much anything they produce is called a "Work of the
Government" and that means, at the federal level, it is
public domain. There are a couple of exceptions and grey
areas, but the basic rule is no copyright.
7 Tales of Bureaucracy
Simple enough: buy video from the government and
upload it. Nobody can stop you. Simple that is, except for
the cost.
But what if we spread the pain out? What if other people
bought some of these tapes and donated them to the
public domain? For $29.95 a month, I signed up for EBay's
ProStores, one of those anybody -can-build-a-store e-
commerce solutions, and built a front-end proxy on top of
the NTIS store. The deal was we'd take your money, order
the tape, upload it to the Internet Archive andYouTube,
and you'd get a tax deduction.
In a fit of marketing, we festooned the site with slogans:
"Be the last person to buy this fine video" and "Buy from
us and you get nothing but everybody gets something."
And, my favorite: "Made by the government — buy in
confidence knowing the source."
OK, so it was a little cheeky and perhaps even a bit
silly. But, the whole business model was silly. With no
intellectual property protection on the content — all of it
works of the government, all already paid for by taxpayer
dollars — if we had enough money we'd simply buy one
copy of each video and we'd be done with it. They'd be out
of business.
The store was snazzy, but there were more "lookie loos"
than buyers. In fact, we got only one order for $106, and
that order was actually a mistake — the guy thought we
were going to send him a DVD.
One day I lost my patience and sent a rather
intemperate fax to the director of the NTIS. A letter is
probably not the appropriate characterization for this
1 Rules for Radicals
communique and maybe flame would be more accurate. I
basically accused the entire of agency of falling down on
the job.
So imagine my surprise the next day when my phone
rang and the voice on the other end said "Mr. Malamud,
this is Ellen Herbst. I'm director of the National Technical
Information Service."
Oh-oh, I thought, here it comes.
Well, Ms. Herbst turned out to be perfectly reasonable.
She wanted the video out there, but by law they were
required to recover their costs and by the time you added
up the people to run the service and factored in the almost
nonexistent sales, well, it cost $70 to sell you a videotape.
If they had to recover their costs, what if we didn't cost
them anything?
"Can you just loan us your videotapes?" I asked the
Director. "You know, send us the tapes, we make a copy, we
send them back to you? We can even pay the postage!"
A long pause. "Yes, I suppose we could do that."
And thus was born FedFlix, where government loans us
tapes which we digitize and send back to them, with a DVD
included for each videotape.
Rule 5 is pretty simple. Keep asking — keep rephrasing
the question until they can say yes.
In November 2007, a couple months after that phone
call — lightening fast by government standards — we signed
a joint venture agreement in which every month NTIS
would send 20 tapes.
We ran that program for a year and put a couple
hundred tapes on-line. At the end of the year, we renewed
7 Tales of Bureaucracy
the agreement and upped the quantity to 100 tapes a
month, and they started sending Betacam masters.
For about $10,000, we've built up a nice little studio that
does professional-quality encoding of the Betacam
masters, producing 8-megabit H.264 MP4 files We don't do
anything fancy like color correcting or normalizing the
sound, but for the content we have that just doesn't seem
necessary.
FedFlix really isn't a funded project, it is something to
fill in gaps in the day. Instead of writing to a Facebook wall,
I choose to rip. I put an egg timer on top of the Betacam
deck, and pop in a tape and set the timer. When it goes off,
I put in another tape.
After a few weeks, there will be a nice collection of
files, I put together a packing list with the metadata, then
use a big old hairy RegEx to turn the packing list into a
bunch of curl calls that use the Internet Archive S3
interface and Python scripts that use the YouTube API.
Perhaps the most subversive thing we do with this
video is put the masters on our server for FTP and rsync.
The hardest part of making a film or a news piece today is
clearing the rights in that absurd thicket of copyright-
obsessed stock footage libraries. With our multi-terabyte
public domain stock footage library, you don't have to ask,
and there are never any late charges in the public domain.
It sometimes amazes me what video gets popular. A
World War II film called "Principles of Refrigeration" has
received 78,000 views. Turns out there isn't any good
HVAC material on the net. The biggest hit on YouTube is a
1 Rules for Radicals
bulldozer safety film called "Stay Calm and Stay in the
Cab!" which has over a half-million views.
— Bureaucracy N° 4 —
One of the big challenges facing government is the deluge
of paper, videotape, and other legacy formats. For
agencies in the information business, such as the Library
of Congress, the National Archives, and many others, the
dual challenges of dealing with legacy formats and how to
face a digital future have been overwhelming. In many
cases, the agencies have turned to what they call public-
private partnerships, so-called "no cost to the
government" deals that have proven to be especially
troublesome.
An example of such a no cost to the government deal
was one cut by the Government Accountability Office, an
arm of Congress, which has the definitive library of federal
legislative history — folders for each public law that contain
all the hearings, bills, and reports that led up to each
statute.
G AO entered into a deal with Thomson West where the
government shipped off all those federal legislative
histories to the vendor, which scanned them and then sent
the paper back.
Not that different than the FedFlix program, but with an
important twist. Thomson West didn't send the GAO back
digital copies of their data. Instead, Thomson West gave
GAO a couple of logins for their staff to use the digitized
material, but for everybody else, including government
7 Tales of Bureaucracy
folks — including congressmen — everybody else has to
pay to access the U.S. federal legislative histories.
The deal wasn't really no cost to the government since
it took a huge amount of effort to pack these 60 million
pages of paper up and send them to the vendor. The
vendor got a sweetheart deal: an exclusive lock on a vitally
important government database. The government got
snookered.
For my next bureaucracy, I want to talk about one of
those public-private partnerships, this one being a deal
that the National Archives cut with Amazon. In December
of 2009, 1 got a call from Congress asking if I could testify
as part of the inaugural hearing for the new Archivist,
David Ferriero.
As part of the research, I looked at the deal the
Archives had cut with Amazon. This was part of Amazon's
new DVD print-on-demand service, and what they had
done was digitize about 1,800 government videos which
they were making available for about $10 per DVD.
I've got nothing against Amazon selling DVDs, even
DVDs of public domain video. But, if you went to the
government site, there was only a 2-minute preview of
each video, in a Microsoft proprietary format, and a
320x240 picture. Next to the 2-minute preview was a
government statement saying you could buy this video
from "our partner" Amazon. Com.
Rick Prelinger — creator of the Prelinger Library and the
real pioneer in rescuing government video — had FOIA'd
the contract behind this arrangement, and it looked like
that while the National Archives got a DVD of their video
1 Rules for Radicals
back, they agreed not to post it on-line for 5 years. There
was a weird arrangement where the government got some
kind of royalty from Amazon, but the royalty was after they
deducted "ingestion fees" for scanning the videos. The
government was paying for the digitization, but wasn't
allowed to use the material.
I asked the Chief of Staff of the National Archives how
much these royalties they were getting were, and it turned
out to be — in 2 years of operation of this partnership — a
total of $3,273.66.
This seemed nuts. So, I forked over $251 and bought 20
DVDs from Amazon and posted them in all the usual
places. Some great stuff, like footage of Richard Nixon in
the White House, explaining why he was innocent of any
wrongdoing.
Then, I wrote to Cory Doctorow at Boing Boing and he
posted a note telling all the happy mutants that if they
watched Richard Nixon onYouTube, they could help save
the public domain because we were counting all the views
to show members of Congress that people really care
about this stuff.
"Watch Richard Nixon, help save the public domain."
The next day, I sent another $461 to Amazon and
ordered another 28 videos, and that led to another Boing
Boing post, "Watch the Bob Hope Christmas Special and
Help Save the Public Domain."
By the time I testified before Congress on December
16, we were able to show more online views for these 48
videos than the total unit sales from the Amazon program
over two years. The message was pretty clear: the Amazon
7 Tales of Bureaucracy
deal had not brought the government any revenue and it
had come at a substantial cost of public access.
We had our point as far as Congress was concerned,
but when I went home I kept looking at those 1,800 videos
and wondered if there was some way to liberate them
without forking over $18,000 to Amazon. I was musing
about this on Twitter and somebody at-replied back and
asked if I had considered an Amazon wish list — the way
you let other people buy stuff for you for your Bar Mitzvah,
birthday, baby shower, or wedding?
Whoa, I thought, what a nice hack!
So, 153 of the most impressive titles went on an Amazon
Wish List and Boing Boing issued a new post suggesting
that if people had an extra $10.95, perhaps they could buy
a Christmas gift for the public domain? (Tax deductible, no
less!)
That list sold out in a matter of days, and the day before
Christmas, my Amazon sent to me, 43 boxes of DVDs. I
spent the holidays ripping the discs, finding metadata, and
uploading files.
It was great. Footage of the Hindenburg blowing up,
James Cagney narrating a cold war film called "the Wall,"
the Cambodian Royal Ballet, old CIA propaganda films,
Disney war films, early space footage, and the Roswell
Area 5 1 investigation.
When you criticize a government agency to their
congressional oversight committee, you're probably going
to get a response. So here is rule 6 for radicals, which is
when you get the microphone, make sure you make your
point clearly and succinctly.
1 Rules for Radicals
Pretty soon, I got a call from the National Archives to
discuss the "Amazon situation." When I said that this video
was totally unavailable to the public, I had misspoke —
anybody could go to the National Archives in College
Park, Maryland and watch any of those 1,800 DVDs onsite.
They'd also let you make a copy of a DVD, and they'd even
furnish the blanks to make those copies — up to 6 copies
per visit.
And, they had more than the 1,800 DVDs in question,
they had over 3,000 DVDs onsite.
"You mean," I asked, "if I went out there often enough, I
could copy all 3,000 of the DVDs and post them?"
"Absolutely. You bet, go for it."
Well, at 10 minutes per DVD, that's 30,000 minutes— 500
hours — more time than I could spend in College Park, but
a perfect opportunity for crowd-sourcing and thus was
born the International Amateur Scanning League.
I wrote to the National Archives chief of staff to give a
courtesy heads-up that I was going to draft a bunch of
volunteers to go out to College Park and systematically
copy all their DVDs. Imagine my surprise when she wrote
back and said David Ferriero thought this was such a great
idea that he'd like to come to the initial meeting of
volunteers and personally teach them how to rip DVDs.
Next thing we knew, we were in a meeting room at the
Sunlight Foundation in the middle of a major blizzard, and
the Archivist of the United States was teaching us how to
rip video. We printed a bunch of red, white, and blue
FedFlix return envelopes for people to send the DVDs as
they finished, and created Public Domain Merit Badges for
7 Tales of Bureaucracy
volunteers who reached certain milestones. If you copy 5
DVDs you get a John F. Kennedy Public Domain Merit
Badge, at 25 discs you get the Bob Hope, and for 50 you get
the Duke Ellington.
— Bureaucracy N° 5 —
Video is really just a hobby for me, something I do in my
spare time. I run a 501(c)(3) nonprofit and we get our
money in the form of grants from foundations such as the
Omidyar Network and corporations such as Google and
Justia.We also get contributions from private foundations
such as the Elbaz, Kapor, and O'Reilly foundations.
Foundations aren't going to give you much money if
your mission statement is "we upload government
videotapes toYouTube."
My day job, as it were, the stuff we're paid to do in the
form of grants and contributions, is to help change our
legal system by making the law more freely available.
You'll remember that with government video, at the
federal level, there is no copyright in works of
government. This principle that there is no copyright is
even more sacred for a protected core — the law. The
principle that we're a nation of laws not a nation of men
means that we write down the rules that citizens must
obey. How can we be a nation of laws if those rules are not
open source?
Despite this principle, access to legal materials in the
United States is a $10 billion per year business. Often,
government will erect barriers to access as a way of
extracting rent from the public.
1 Rules for Radicals
This is particularly true for a database run by the
Administrative Office of the U.S. Courts, a database called
PACER, which stands for Public Access to Court Electronic
Records. PACER contains 500 million pages of the
proceedings of the U.S. district courts, including the
dockets, briefs, motions, and opinions of every U.S. federal
case.
The courts charge 8 cents per page and require a valid
credit card to access PACER. A prisoner or other citizen
can petition a judge for free access. But, petitioning a
federal judge isn't exactly a low barrier to entry.
This is a big business for the courts: they drag in $120
million a year in revenue. The courts even charge the
executive branch of the federal government millions a
year to access filings!
To poke a few fingers in the eyes of the Administrative
Office, we put up a recycling site, which let people upload
their PDF documents from PACER — where we'd recycle
them into the public domain.
Since PACER is a half-billion-page database, it was
really kind of a bluff, a vehicle for an FAQ that tried to
expose the finances behind PACER. But one of the things in
the FAQ caught the attention of a couple of volunteers.
You see, the Courts, under strong congressional
pressure to do something about public access, had just
launched a trial program, putting one terminal in each of
17 libraries around the country. In the FAQ for the PACER
recycling site, I encouraged volunteers to join the so-
called Thumb Drive Corps and download docs from the
7 Tales of Bureaucracy
public access libraries and upload them to the PACER
recycling site.
Aaron Swartz, whom many of you may know as the
editor of the RSS spec and a prolific contributor to the
Internet, called up and said he'd like to join the Thumb
Drive Corps. I told him to be careful, knowing he was
technically astute and inclined to script things pretty, um,
aggressively. I warned him to make sure he didn't violate
any of the guidelines the courts had set: if they said don't
download too many docs, don't download too many docs.
A few weeks later I got email saying he had some data,
could he maybe get an account to upload his docs
directly? Sure, no problem, we let him SSH in, and data
started to come in, and come in, and come in, and soon
there were 760 gigabytes of PACER docs, about 20 million
pages.
Aaron evidently had super-sized his Thumb Drive, but
he's a bright guy, so we weren't totally surprised.
Then, the stream abruptly stopped and I got email from
Aaron saying we needed to talk. Right away.
The Administrative Office had evidently finally looked
at their usage logs after two months — and then abruptly
cancelled the public access program overnight, saying a
security breach had occurred. The Superintendent of
Documents at the Government Printing Office gave a
speech and said that not only had a security breach
occurred, the FBI had been called in to investigate.
Aaron and I talked again, and after grilling him, I was
still convinced we had done nothing wrong. There were no
signs or appropriate use statements saying this was
1 Rules for Radicals
intended for casual use only. I'll grant you that 20 million
pages had perhaps exceeded the expectations of the
people running the pilot public access project, but
surprising a bureaucrat isn't illegal.
From previous experience putting Court of Appeals
decisions on-line, I was pretty sure this PACER data was
going to be a mess. Rather than release the data on the net,
I started an audit looking for privacy violations.
For the next two months, a series of scripts ran that
looked for personal identifiers. Any files with a hit were
manually examined. Many of them were false positives,
such as government contract numbers.
But, there were also a whole bunch of files that did have
problems, and for each of those I looked around for things
the regex didn't catch and ended up finding even more
Social Security numbers, and other illegal data like the
names of minors and bank account numbers.
There was the obvious stuff, like the IRS suing a citizen
and forgetting to redact their Social Security number on
tax returns filed as evidence. Or, redacting the number by
placing a black rectangle on top of the text or turning the
color of the text to white.
There was also some really heart-wrenching stuff, like a
list of 350 patients of a doctor who was being sued for
malpractice. For each patient, the supporting document
listed their home address, birth date, Social Security
number, and a list of all their medical problems. Or the list
of the members of labor unions involved in pension
disputes, with their personal identifying information, home
address, and earnings history.
7 Tales of Bureaucracy
After completing that analysis, we sent a formal audit
over to the Administrative Office of the Courts with a
carbon copy to the judge who chairs the Judicial
Conference Rules Committee. In addition to a printed list,
they got a DVD that let them compare the redacted to the
un-redacted version of 2,000 offending documents.
You'd think this was pretty shocking evidence, but the
Administrative Office of the Courts ignored the
preliminary audit, then ignored the final audit, then
continued to ignore us. Finally, over the Christmas holidays
in 2008, letters went to the Chief Judges of 30 district
courts.
On the top of those letters — in big red type — were the
words "Third and Final Notice." The letters said we had
sent a preliminary audit to the Administrative Office and a
final audit to the Administrative Office, and of course, these
letters said, it goes without saying that the Administrative
Office had promptly notified the judges of these very
serious problems, since of course they didn't want to be
breaking the law.
Needless to say, the judges hadn't heard about this
situation, and you have to swallow real hard before you
send Chief Judges of U.S. District Courts letters saying
"Third and Final Notice," but you know what ... judges are
reasonable people. They got these letters, their clerks
checked them out, and we started getting letters back
saying in effect "you're right, thanks, we'll take steps."
As a result of those audits, the Senate sent a strongly
worded letter to the Administrative Office asking them why
they weren't obeying the law. The judge who chairs the
1 Rules for Radicals
Rules Committee wrote back to the Senate saying that
while they had privacy rules in place, they were obviously
not going far enough and they would change their rules.
And, a few months later, they changed their rules.
Here is my seventh rule for radicals, which is to get
standing. One can criticize government all one wants, and
they'll often ignore you. But, if there is something clearly
wrong and against the law and you can document that
malfeasance and wrongdoing, they have to talk to you. If
you have standing, you can insist.
— Bureaucracy N° 6 —
There is a related rule, and that is rule 8, which is to try to
get the bureaucrats to threaten you. Remember how the
law has a special place when it comes to copyright? While
a state government might be able to assert copyright over
some things, the Supreme Court has repeatedly ruled that
nobody can copyright the law. This means no copyright on
court opinions, but it also means no copyright on state
statutes.
So, you can imagine our surprise when the Oregon
Legislative Counsel, the lawyers for the legislature of
Oregon, sent a takedown notice to Public.Resource.Org
and to Justia, a company that has been instrumental in
putting free law on-line. The state said that by making
tarballs of the 2007 Oregon Revised Statutes available for
anonymous FTP, we had violated their copyright.
Why would the Oregon legislature insist on copyright?
Money! They sold a print edition and they made money on
7 Tales of Bureaucracy
that print edition. We were threatening their revenue
stream.
Now, to be totally fair, the policy in question had been
put in place in the mid 1940s and nobody had ever
questioned that policy. The takedown notice was the action
of a bureaucrat just doing what they'd been doing for 70
years.
Once you have a takedown notice, particularly from a
body as eminent as the lawyer for the Oregon legislature,
you are in legal peril. You have a right to think they're
going to sue you, because that's what the takedown notice
says.
If you're in legal peril, you can go to a judge and ask for
what's called declaratory relief, asking the court to rule on
the issue. So, we hired a lawyer and put together a draft
declaratory relief request and posted it on the net.
The thing about a state sending you a takedown notice
for putting the law on the Internet is that this is not one of
the subtle legal issues that you have to carefully explain to
people. Everybody gets this — you can walk into any bar
and explain what's going on and everybody will instantly
get the issue and say "that's really stupid."
And that's my rule 9 for radicals — look for over-
reaching, something that is clearly nuts. Not being able to
publish state statutes certainly qualified on that count.
A few days after we threatened to sue, we got a notice
saying the Oregon Legislature had scheduled hearings on
this issue and would we be prepared to testify? Of course!
If they were willing to talk, we certainly were.
1 Rules for Radicals
We testified, as did some Oregon citizens. The lawyer
for the legislature gave his testimony, and I was impressed
by how well informed and willing to look at the issues
everybody was.
After a lot of questions, the Legislative Counsel
Committee, which was chaired by the President of the
Senate and the Speaker of the House, voted unanimously to
waive any assertions of copyright. It was democracy in
action, and way quicker than a law suit.
To prove the point about why this was so important to
do, a few months later, a second-year law student at Lewis
& Clark took the statutes and created OregonLaws.Org, a
dramatically better version of the Oregon statutes
featuring a great UI, valid HTML, permaURLs, an iPhone
app, tag clouds, a twitter feed, and loads of other bells and
whistles.
When a state asserts copyright over legal materials, it
is important to remember that while this is partly about
democracy and justice, it is also about innovation. By
requiring a license as a precondition to access primary
legal materials, we create a barrier to innovation.
— Bureaucracy N° 7 —
I'd like to end this tale with a bureaucracy that is a bit
amorphous, a little hard to visualize and thus an
exceedingly difficult target and that bureaucracy is all the
lawyers in the United States of America. When it comes to
bureaucracies, the bar truly is the borg.
The principle that access to the law must be unfettered
is a basic foundation of our system of justice. The U.S.
7 Tales of Bureaucracy
constitution says that "equal protection under the law" may
not be denied. Equal protection means that your basic
rights cannot be arbitrarily denied because you are poor,
are of a certain religion or race, or because somebody
disagrees with your political views.
A poll tax, which preconditions access to the polls on
access to money, is wrong because it denies equal
protection under the law. I put it to you that just as a poll
tax is wrong, preconditioning access to primary legal
materials on having a credit card is just as wrong, and it
violates our rights to equal protection under the law.
Turning primary legal materials from public property
into private parcels violates more than equal protection, it
makes due process under the law impossible, when rich
lawyers can do more research than poor lawyers.
By poor lawyers, I mean public interest lawyers and
solo practitioners. I also mean government lawyers in
places like the Department of Justice, who, believe it or
not, get memos telling them to please stop doing so much
research because the department is over budget.
Going back to the 1824 decision in Wheaton v. Peters,
one of the landmark cases of the great Marshall Court, the
Supreme Court, has been clear, over and over again, that
there is no copyright over primary legal materials, be they
court opinions or administrative regulations or state
statutes or OSHA regulations or even building codes
drafted by third parties but duly enacted as the law of the
land.
Despite this clear public policy, states and
municipalities have erected a thicket of copyright
1 Rules for Radicals
restrictions, pay walls, and click-through contracts around
the raw materials of our democracy.
How do you change something so basic, so
fundamental as access to the law?
This year, people involved in the free law movement
have been gathering together under the banner of the
Law.Gov initiative, an effort to try and convince policy
makers from water districts to the President and Chief
Justice that access to primary legal materials matters.
Our strategy to get this basic principle — that access to
primary legal materials must be unfettered and reliable,
must be available in bulk, and cannot be subject to pay
walls or copyright restrictions — has started with a national
conversation, a series of working groups and workshops
held in many of the top law schools in the country.
We are in the middle of this process right now, in fact
have just completed our workshop at Duke Law School's
Center for the Public Domain, a workshop that featured not
only well known legal scholars, but had the active
participation of the Archivist of the United States and
Andrew McLaughlin, who is Deputy Chief Technology
Officer in the Executive Office of the President.
We will be completing this process of workshops and
working groups in June, and will issue a set of
recommendations to government officials. The Ninth
Circuit of the Court of Appeals has granted us time to brief
the judges, the U.S. Senate has asked for a copy of the
report, and I've been very impressed at the number of top
administration officials, members of Congress, and Chief
7 Tales of Bureaucracy
Judges in the Judicial Conference who have taken a keen
interest in these proceedings.
— 10 Rules for Radicals —
In this talk, I've tried to present to you some rules for
radicals, some techniques that I use in my work, but
techniques that perhaps might be useful in your own
efforts to change how institutions function. We've covered
nine of those rules so far. Let me recap.
Rule 1 : Call everything an experiment.
Rule 2: When the starting gun goes off, run really fast.
As a small player, the elephant can step on you, but you
can outrun the elephant.
Rule 3: Eyeballs rule. If a million people use your
service, and on the Internet you can do that, you've got a
lot more credibility than if you're just issuing position
papers and flaming the man.
Rule 4: When the time comes, be nice.
Rule 5: Keep asking until they say yes. Gordon Bell, the
inventor of the VAX, once said that you should keep your
vision, but modify your plan.
Rule 6: When you get the microphone, get to the point.
Be clear about what you want.
Rule 7: Get standing. Have some skin in the game, some
reason you're at the table.
Rule 8: Get them to threaten you.
Rule 9: Look for overreaching, things that are just
blatantly, obviously wrong or silly.
And finally, rule 10, which is don't be afraid to fail. It
took Thomas Edison 10,000 times before he got the
1 Rules for Radicals
lightbulb right, and when he was asked about those
failures, he said "I have not failed, I've just found 10,000
ways that won't work."
Fail. Fail often. And don't forget, you can question
authority.
Thank you very much.